Skip to content

chore: fetch Logpush dataset pages from middlecache - #33401

Open
mvvmm wants to merge 3 commits into
productionfrom
fetch-logpush-datasets
Open

chore: fetch Logpush dataset pages from middlecache#33401
mvvmm wants to merge 3 commits into
productionfrom
fetch-logpush-datasets

Conversation

@mvvmm

@mvvmm mvvmm commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds bin/fetch-logpush-datasets.ts, which downloads the generated Logpush dataset field-definition pages (zone/*.md, account/*.md) from middlecache and syncs them into src/content/docs/logs/logpush/logpush-job/datasets/. This follows the existing skills/ middlecache pattern:

  • Fetches on prebuild (fetch:assets) and predev (--soft)
  • Supports --force for manual re-fetch (pnpm fetch:logpush) and MIDDLECACHE_BASE_URL override
  • The 40 generated .md pages are now gitignored and no longer committed; the hand-written .mdx files (index.mdx ×3, cmb.mdx) remain tracked
  • Reuses downloadToDotTempIfNotPresent + extractTarGz (retry, integrity check, zip-slip protection)

Replaces the periodic manual "[Logs] Update Logpush dataset field definitions" PRs with a build-time fetch from middlecache.

@github-actions github-actions Bot added product:logs Related to Logs size/xl labels Sep 11, 2026
@mvvmm
mvvmm marked this pull request as ready for review September 11, 2026 23:13
@mvvmm
mvvmm requested review from a team, Sahidya, rianvdm and soheiokamoto as code owners September 11, 2026 23:13
@cloudflare-docs-bot

cloudflare-docs-bot Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Review

✅ No issues found in commit d16f2fe.

Code Review

This code review is in beta and may not always be helpful — use your judgment.

No code review issues found.

Conventions

No convention issues found.

Style Guide Review

No style-guide issues found.

Commands

Only codeowners can run commands. Post a comment with the command to trigger it.

Command Description
/review Runs a review now. Incremental if a prior review exists, full if not.
/full-review Re-reviews the entire PR diff from scratch, ignoring incremental history. Useful after a rebase, when you want a fresh review, or if the bot gets out of sync and reports issues that no longer exist.
/ignore-review-limit Permanently lifts the 2-review automatic limit for this PR. Future pushes will trigger reviews as normal.
/disable-auto-review Stops automatic reviews from triggering on future pushes to this PR. Codeowners can still run /review or /full-review manually.
/rebase Rebases the PR branch against production. On conflict, attempts to resolve automatically using AI. Stops with an explanation if confidence is not high enough.

@github-actions

Copy link
Copy Markdown
Contributor

This pull request requires reviews from CODEOWNERS as it changes files that match the following patterns:

Pattern Owners
* @cloudflare/product-owners
*.ts @cloudflare/content-engineering
package.json @cloudflare/content-engineering
/src/content/docs/logs/ @angelampcosta, @rianvdm, @sahidya, @soheiokamoto, @cloudflare/product-owners

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 11, 2026

Copy link
Copy Markdown

🚀 Deploying Preview to Cloudflare 🚀

Preview URL: https://fetch-logpush-datasets.previews.developers.cloudflare.com (commit d16f2fe)

This URL reflects your latest Preview deployment

Preview Deployments by commit

Status Deployment URL Commit Updated (UTC) See this deployment's details
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://f98f5d4b.previews.developers.cloudflare.com d16f2fe 2026-09-11T23:46:29.018Z Visit the dashboard ↗
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://6008cb1f.previews.developers.cloudflare.com b00615a 2026-09-11T23:31:59.425Z Visit the dashboard ↗

@github-actions

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants